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Abstract 

This paper starts a systematic study of capacity-achieving sequences of low-density parity- 
check codes for the erasure channel. We introduce a class A of analytic functions and develop a 
procedure to obtain degree distributions for the codes. We show various properties of this class 
which will help us construct new distributions from old ones. We then study certain types of 
capacity-achieving sequences and introduce new measures for their optimality. For instance, it 
turns out that the right-regular sequence is capacity-achieving in a much stronger sense than, 
e.g., the Tornado sequence. This also explains why numerical optimization techniques tend to 
favor graphs with only one degree of check nodes. Using our methods, we attack the problem 
of reducing the fraction of degree 2 variable nodes, which has important practical implications. 
It turns out that one can produce capacity achieving sequences for which this fraction remains 
below any constant, albeit at the price of slower convergence to capacity. 
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1 Introduction 



Low-density parity-check codes have attracted a lot of attention lately. Very simple and efficient 
decoding algorithms and the near capacity performance of the codes with respect to these algorithms 
have made them one of the most powerful classes of codes known to date. Despite recent advances 
in the asymptotic analysis of these codes [1, 2, 3], for all nontrivial channels except for the erasure 
channel it is still unknown whether there exist sequences of these codes that meet the Shannon 
capacity. The case of the erasure channel is the simplest to analyse, and a thorough understanding 
of this case seems to be a prerequisite for understanding the more general situation. Moreover, [3] 
showed that many concepts that were first developed for the erasure channel carry over to the case 
of other more complicated channels. For this reason, we will start in this paper a systematic study 
of capacity achieving sequences of low-density codes over the erasure channel. 

In [4] the authors introduced a simple algorithm for correcting erasures in a low-density parity- 
check code. To describe the result, we need some piece of notation. We visualize low-density 
parity-check codes as bipartite graphs between a set of left nodes called variable nodes and a 
set of right nodes called check nodes. An edge in this graph is said to have left-degree i if it is 
connected to a variable node of degree i. Similarly, it has right-degree i if it is connected to a 
check node of degree i. Let A, and pi denote the fraction of edges of left-degree and right-degree i, 
respectively. A degree distribution for the graph is then the pair (A, p), where A = A(x) = £\ 
and p = p(x) = J2iPi x% ~ 1 - The main result of [4] states that their simple recovery algorithm is 
successful on a random graph with degree distribution (A,p) and initial erasure probability 6 if 

6\{l-p{l-x)) <x (1) 

on the interval (0, S). In this case, we say that the pair (A, p) affords 5. The capacity of the erasure 
channel with probability S is given by 1 — 6. On the other hand, (A,p) determine the rate of the 
code as R = 1 - f$ p(x)dx/ Jq \(x)dx. We will call R the rate of the pair (A,p). The first design 
goal is thus to produce pairs (A,p) such that the maximum 6 that satisfies (1) is very close to 
1 — R. We call a sequence (A",p n ) of degree distributions capacity-achieving (c.a. for short) if the 
maximum 6 n afforded by (A n ,p n ) converges to its uper bound 1 - R as n tends to infinity. 

How can we produce c.a. distributions (A n ,p n )? A closer study of two examples of c.a. distributions 
in the literature is very helpful. For the first sequence, called the Tornado sequence, X n (x) is the 
initial segment of the series — ln(l — x) properly normalized to give A n (l) = 1, and p n (x) is the 
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initial segment of an exponential exp(/i(l - x)), where is computed from the rate-constraint [4]. 
For the second sequence, called the right-regular sequence, we have p n {x) = x n and X n (x) is related 
to the power series (1 - x) l l m for some m [5]. Roughly speaking, in both cases we start with 
a f(x) represented by a Taylor series with non-negative coefficients on [0, 1] and satisfying the 
normalizations /(0) = 0, /(l) = 1, for which 

Tf(x):=l-rHl-x) (2) 

has again a converging Taylor series expansion with non-negative coefficients. The existence of the 
inverse function / _1 needed in (2) is automatic from the conditions. We therefore define the set 

V := {/(*) = £ f k x k , xe [0, 1] | f h > 0, /(I) = 1}, (3) 
l 

and the set A as the maximal subset of V invariant under the action of T: 

A:={feV\Tf€V}. (4) 

Later in Section 2 we will introduce a general method for obtaining degree distributions from : 
elements of A. The nonlinear operator T has two interesting properties: first, its square is the * 
identity, and second, it commutes with composition o in the sense that T(f o g) = Tg o Tf. Using 
these identities, we will be able to construct infinitely many c.a. sequences starting from known , 
ones. 

^Prom a practical point of view achieving capacity is not sufficient. Suppose that (A n ,p n ) is a * 
sequence of degree distributions of rate R. What we would like to know is how fast, if at all, the * 
maximal 5 = S n afforded by (A n ,p n ) converges to 1 - R as n -¥ oo. This problem was studied , 
in [5], where it was shown that if a r is the average degree of the check nodes (l/o r = Jq p(x)dx) 
and e := 1 - 5/(1 - R), where 8 is afforded by (A,p), then a T > log(e)/log(.R), or, equivalently, 
e > R ar . In that paper a sequence of degree distributions is called asymptotically quasi-optimal \ia T 
stays bounded by jilog(l/e) where /z is a constant depending on R. It was shown that the Tornado 
sequence and the right-regular sequence are both asymptotically quasi-optimal. A much more 
refined notion of optimality would be to directly compare R ar and e. We call a sequence of degree 
distributions asymptotically optimal if e/R ar stays bounded by some constant depending on the 
rate. Obviously, asymptotical optimality implies asymptotical quasi-optimality. Using this notion, 
we will show later in Section 3 that the right-regular sequence is asymptotically optimal while 
the Tornado sequence turns out to be only asymptotically quasi-optimal. This result may explain 
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why numerical optimzation techniques [7] tend to produce sequences that are close to being right 
regular. Given that this also holds for other types of channels [3], we conjecture that right regular 
sequences are fundamentally superior on symmetric channels. Using the composition operation, we 
will also construct infinitely many sequences of asymptotically quasi-optimal degree distributions. 
It is proved in [6] that for a c.a. sequence (A n ,p n ) the sequence S n \ n {l - p n {l - x)) - x and all 
its derivatives up to any fixed order k converge uniformly to zero in [0, 1) as n goes to infinity. 
Examining the first derivative, this shows that the product 5 n \%(p n Y{l) converges to 1 as n goes 
to infinity. As a result, no c.a. sequence has the property that for all sufficiently large n we have 
A" = 0. On the other hand, from a practical point of view it is advantageous to have as little degree 
2 variable nodes as possible, since these are the nodes that get corrected last. In particular, if the 
fraction of degree 2 nodes is at most 1 — i?, then one can construct the graph in such a way that 
all the redundant symbols fall within the set of degree 2 variable nodes, and one does not need to 
correct these nodes at all. This greatly accelerates the decoding procedure. Using our techniques, 
we will construct in Section 4 asymptotically quasi-optimal sequences of degree distributions whose : 
fraction of degree 2 variable nodes is strictly less than 1 — i?. 

In this extended abstract we have deliberately omitted the proofe, and numerical evidence in support 
of our theorems. They can be found in the final version of the paper which can be obtained from 
the authors upon request. 

2 The class A 

We start with more details on the properties of A defined by (4). Note that by definition / E A 
implies absolute convergence of the Taylor series of both / and T/. It is straightforward to check 
that T is an involution, i.e., T 2 / = /, that if / G A then S a j,f{x) := f ft*+$~J{§ G A for any 
0<a<a + 6<l, and, finally, that for any / G ?, we have J* Tf(x) dx = J* f{x) dx. Further, it 
is easy to check that each of the families of functions 

f(x) = x n , n = l,2,..., /(*) = £_, o>0, /(«) = L_JL, 0<l<1 - ^ 

belong to A. The first two families correspond to the right-regular and Tornado sequences, while 
(5) has the property Tf = /; we call that the self-inverse sequence. Many more examples can be 
constructed from these families by using the above properties of A. Below, we will systematically 
explore sequences of the form f(x) = <£(x n ), n = 1, 2, . . . , generated by composition of some <f> e A 
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with the right-regular sequence. More examples can be found in later sections. 

What we outline next is how to produce for given / G A and R G (0, 1) a suitable pair of left- and 
right-degree distributions (A, p) of rate R. Roughly speaking, p and A are defined by scaled sections 
of the Taylor expansion of / and T/, respectively. Throughout the paper, we use the notation [*j 
for the integer part of a real number t y and set {t} — t — |_*J € [0, 1). If f(x) = 5^*>i A x * £ 
then we set 

Ttf(x) := £ + {*}/ W+l *W , f t f(x) := p£g> (6 ) 
*<UJ 

for the Taylor polynomial T t f of degree [£j -f- 1 > 0 of / G A and its normalization defined by 
Ttf{\) — 1, respectively. Our algorithm for computing the pair (A,p) is as follows. 

Algorithm 1. Given f G A, R G (0,1), and an integer N, this algorithm computes a pair (A,p) 
of rate R and a real number S such that p(x) is of degree N and (X, p) affords 5. 

(1) If f(x) is a polyomial, then set p(x) := f {x), otherwise set p(x) := T/v/(x). Let a r := 

(2) Let g[x) := Tf(x) - Zk>x 9k^ k - Find t such that jj f t ^(x)cb = (iz^. 

(3) Set S := T t g{l) and \{x) := T t g{x)/S. 

The final version of the paper states clearly what Step (2) means from a computational point of 
view. It also proves that if f(x) is analytic at 1, and a r computed in Step (1) of the above algorithm 
satisfies (1 — R)a T > 2, then the algorithm correctly computes its output, i.e., the output satisfies 
the specifications of the algorithm. The running time of the algorithm depends heavily on how long 
it takes to compute all the coefficients of necessary. This depends ultimately on the function / 
and the convergence behavior of its Taylor coefficients. For most of our applications, however, we 
have analytic expressions for the which makes it rather trivial to estimate the right value of t. 

3 Convergence speed of c.a. sequences 

This section is devoted to estimating how close the performance of the decoding algorithm for the 
code given by the pair (A, p) is to the capacity of the erasure channel. In other words, we want to 
study how close the maximal 6 afforded by this pair is to 1 - R where R is the rate of the pair. We 
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introduce e(A,p) := 1 - ^r^- As stated in the introduction, it is proved in [5] that e(\,p) > R" r 
where l/a r = J* p(x)dx. This shows, that in order to have e -> 0 or, equivalently, 1 — 5 -» /?, we 
must have a r oo for fixed R. Also, since e > R ar , we may use either 

x a r log(R) A/ . x e(A,p) 

to quantitatively measure the closeness of capacity and rate of any particular pair (A,p), hence 
also of c.a. sequences. In refining the definition introduced in [5], we will call a sequence (A n ,p n ) 
asymptotically quasi- optimal with constant p > 1 if limsup n ^ OG /z(A n ,p n ) = p. Clearly, the closer p 
is to 1 the better it is. In addition, we call a sequence (A n ,p n ) asymptotically optimal with constant 
A > 1 if limsup n _^ 00 A(A n ,p n ) = A. Trivially, asymptotically optimal sequences with any A are 
asymptotically quasi-optimal with constant 1. As it turns out these definitions allow to better 
classify c.a. sequences, and see distinctive differences between, e.g., the Tornado and right-regular 
sequences. 

Let </> € A. We start with sequences (A n , p n ) of fixed rate R G (0, 1) generated from the sequence 

f n (x):=<t>(x n ) } n>n 0 , ' (8) 

by Algorithm 1. For short, we say that this sequence is generated by <j>. Since </> G A all f n satisfy , 
the additional analyticity condition at x = 1. If ^ is a polynomial of degree K then we choose 
N = Kn and set p n = / n ; otherwise N = N(n) is such that T N f n (l) > 1/(1 + f n {R)). 

Theorem 1. // a sequence (X n y p n ) of rate R is generated by </> € A as described above, then it is 
asymptotically quasi-optimal with constant p, < p? := rl }( x \ — . 

Jo x d:C 

We remark that Theorem 1 provides only an upper bound for the constant p of quasi-optimality. 
However, numerical evidence strongly suggests that the bound is rather sharp. The following result 
addresses the Tornado and the right-regular sequences and gives a more quantitative statement on 
their asymptotic quasi-optimality previously established in [4, 5]. 

Theorem 2. (1) Let (A a ,p a ) be the sequence of degree distributions of rate R E (0,1) obtained 
from the family f a (x) = (e ax — l)/(e° — 1), a -> oo, by Algorithm 1. This sequence is 
asymptotically quasi-optimal with constant p < pTom{R) •= i^^- 

(2) Let (A n ,p n ) be the right-regular sequence of degree distributions of rate R E (0,1) obtained 
from the family (5) by Algorithm 1. This sequence is asymptotically optimal with constant 
A = e 7 - 1.78107241..., where 7 = 0.57721566... is the Euler constant 
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In contrast to the bounds in Theorem 1, the bound of Theorem 2(1) is rate-dependent. Note that 
MTom(i?) -> oo as R -> 0 and fiTom(R) -> 1 as 1. As a final comment, we mention that 
the Tornado family does not produce an asymptotically optimal c.a. sequence. Compared with the 
statements before it and in light of the numerical evidence, Theorem 2(2) exhibits the excellent 
asymptotic behavior of the right-regular sequence. We do not know of any asymptotically optimal 
c.a. sequence with a constant A < e 7 . 

4 Reduction of degree 2 nodes 

In this short section, we concentrate on the asympotic behavior of the fraction of degree 2 variable 
nodes of the c.a. sequences generated by the family (8). As discussed in the introduction, this frac- 
tion should be as small as possible to allow fast decoding. For an arbitrary left-degree distribution 
A, it is defined by the coefficient A2 of the Taylor series 

4454? = A 2* 2 + A 3 * 3 + .... The stability 

Jo H x )<*x 

condition [5, 3] states in the case of the erasure channel that if (A,p) affords J, then SX^p^l) < 1, 
where X(x) = A 2 x + 0 (x 2 ). Hence, we have A 2 = (1 "^ QrA2 < sfeffijTJ - If we a PP J y thi s inequality 
to a c.a. sequence {X n ,p n ) of rate R, then using S n == J(A n ,p n ) -» 1 - R we obtain 

n 1 ^ A " * £& 2(^(1) ' ^ = (l ^ (X)da: ) 1 ' (9) 
if the limits exist (otherwise, the relation holds at least with lim replaced by limsup or liminf). 
Moreover, if the c.a. sequence (A n ,p n ) of rate R is generated by (/> G A as explained in Section 3, 
then we can replace p by / and prove the existence of the limits and equality in (9). 

Theorem 3. (1) // the sequence (A n ,p n ) of rate R is generated by <f> e A, then lim n _>oo A£ = 

(2) For any 0 < R < 1, Mere extste o c.a. sequence (A n ,p n ) of rate R which is asymptotically 
quasi-optimal and has a limit lim^oo A£ < 1 - R. 

The proof of part (2) of the above theorem is obtained by studying <f>(x n ) for some <f> G A. The 
final version of the paper contains tables that illustrate the quality of the asymptotic estimates in 
Theorems 1-3. 
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